United States Patent and Trademark Ofhce 



UNITED STATES DEPARTMENT OF COMMERCE 
United States Patent and Trademark OtBce 

Address: COMMISSIONER FOR PATENTS 



APPLICATION NO. 



FILING DATE 



FIRST NAMED INVENTOR 



ATTORNEY DOCKET NO. CONFIRMATION NO. 



10/678,043 



10/01/2003 



John W. Mcintosh 



MclntoshProgrammerator 



27119 7590 11/13/2008 

ALBERT W. WATKINS 
30844 NE 1ST AVENUE 
ST. JOSEPH, MN 56374 



HEFFINGTON, JOHN M 



PAPER NUMBER 



DELIVERY MODE 



Please find below and/or attached an Office communication concerning this application or proceeding. 

The time period for reply, if any, is set in the attached communication. 



PTOL-90A (Rev. 04/07) 



KJttiVrXi nvrliyjts OUff Iff fcff Jr 


Application No. 

10/678,043 


Applicant(s) 

MCINTOSH ETAL. 


Examiner 

JOHN M. HEFFINGTON 


Art Unit 
2179 





- The MAILING DATE of this communication appears on the cover sheet with the correspondence address — 
Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) OR THIRTY (30) DAYS, 
WHICHEVER IS LONGER, FROM THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
eamed patent term adjustment. See 37 CFR 1 .704(b). 

Status 

1 )^ Responsive to communication(s) filed on 22 August 2008 . 
2a )□ This action is FINAL. 2b)|3 This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) ^ Clalm(s) 8-21 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) \Z\ Claim(s) is/are allowed. 

6) |EI Claim(s) 8:21_ is/are rejected. 
/)□ Claim(s) is/are objected to. 

8) 0 Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) 0 The specification is objected to by the Examiner. 

10) 0 The drawing(s) filed on is/are: a)^ accepted or b)^ objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held In abeyance. See 37 CFR 1.85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

1 1) 0 The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12) 0 Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 
a)n All b)n Some * c)^ None of: 

1 .□ Certified copies of the priority documents have been received. 

2. n Certified copies of the priority documents have been received in Application No. . 

3. n Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 



Attach ment(s) 

1 ) ^ Notice of References Cited (PTO-892) 4) □ Interview Summary (PTO-41 3) 

2) □ Notice of Draftspereon's Patent Drawing Review (PTO-948) Paper No(s)/IVIail Date. 

3) □ Information Disclosure Statement(s) (PTO/SB/08) 5) □ Notice of Informal Patent Application 

Paper No(s)/Mail Date . 6) □ Other: . 



PTOL-T26'(Rev^'o8-0^^ 



Office Action Summary 



Part of Paper No./Mail Date 20081 104 



Application/Control Number: 10/678,043 Page 2 

Art Unit: 2179 

DETAILED ACTION 

This action is in response to the Request For Continued Examination filed 22 August 
2008. Claims 1-7 have been canceled. Claims 8, 14 and 15 have been amended. 
Claims 16-21 are new. Claims 8-21 are pending and have been considered below. 

Continued Examination Under 37 CFR 1.114 

1 . A request for continued examination under 37 CFR 1.114, including the fee set 
forth in 37 CFR 1 .17(e), was filed in this application after final rejection. Since this 
application is eligible for continued examination under 37 CFR 1.114, and the fee set 
forth in 37 CFR 1 .17(e) has been timely paid, the finality of the previous Office action 
has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 22 
August has been entered. 

Response to Arguments 

2. Applicant's arguments with respect to claims 8-21 have been considered but are 
moot in view of the new ground(s) of rejection. 

Claim Rejections - 35 USC § 103 

3. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the phor art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary sl<ill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 
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4. Claims 8-21 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Tuttle et al. (US 5,157,782) in view of Moore et al. (US 5,604,509). 



Claim 1: 


(Canceled) 


Claim 2: 


(Canceled) 


Claim 3: 


(Canceled) 


Claim 4: 


(Canceled) 


Claim 5: 


(Canceled) 


Claim 6: 


(Canceled) 


Claim 7: 


(Canceled) 



Claims 8 and 16: Tuttle discloses method for using a first computer system to remotely 
monitor and interact with the operation of a second computer system through a 
graphical user interface of said second computer system, comprising the steps of: 

a. receiving a first graphical element or a pixel image of a first graphical element of 
said second computer system graphical user interface at said first computer 
system (column 6, lines 11-21, column 8, lines 39-47); 

b. generating a user peripheral input device input action within said second 
computer system graphical user interface as interpreted by a second computer 
peripheral input device controller channel by passing a signal from said first 
computer system to said second computer system graphical user interface 
(column 6, lines 1-10, 22-28); 
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c. monitoring said second computer system graphical user interface or a pixel 
image of said second computer system graphical user interface from said first 
computer system for an expected second graphical element (column 6, lines 28- 
30, 34-39); and 

d. signaling a failure at said first computer system if said expected second graphical 
element is not detected (column 6, lines 39-41). 

but does not disclose 

a. generating a user peripheral input device input action within said second 
computer system graphical user interface as interpreted by a second computer 
peripheral input device controller channel by passing a signal through an i/o 
communications channel from said first computer system to said second 
computer system graphical user Interface responsive to said receiving step; 

b. monitoring said second computer system graphical user interface or a pixel 
image of said second computer system graphical user interface from said first 
computer system for an expected second graphical element within a 
predetermined time interval; and 

c. signaling a failure at said first computer system if said predetermined time 
interval elapses without detecting said expected second graphical element. 

However, Moore discloses a system where video data is sent from a second computer 
(a system under test) to a first computer (host or remote computer) over a 
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communication cliannel (column 2, lines 41-59) and displayed on a display of the first 
computer (column 3, lines 8-19). A user inputs, i.e. mouse and keyboard inputs, can be 
sent over a communications channel from the first (host or remote computer) to the 
second computer, system under test (column 4, lines 16-23) where the input data is 
processed (column 4, lines 39-50). Therefore, it would have been obvious to one 
having ordinary skill in the art at the time of the invention to add generating a user 
peripheral input device input action within said second computer system (system under 
test) graphical user interface as interpreted by a second computer peripheral input 
device controller channel by passing a signal through an i/o communications 
channel from said first computer system (host or remote computer) to said second 
computer system graphical user interface responsive to said receiving step to Tuttle. 
One could have been motivated to add generating a user peripheral input device input 
action within said second computer system graphical user interface as interpreted by a 
second computer peripheral input device controller channel by passing a signal 
through an i/o communications channel from said first computer system to said 
second computer system graphical user interface responsive to said receiving step to 
Tuttle in order to extend the range of communication between the first and second 
computers of Tuttle. It appears that Tuttle is intended to be used wherein the first and 
second computers are in close proximity. Adding the limitations of Moore to Tuttle could 
allow Tuttle to be able to test the software/hardware of the second computer at the first 
computer over a network over a larger distance. 
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Neither Tuttle nor Moore disclose generating a user peripheral input device input action 
within said second computer system graphical user interface as interpreted by a second 
computer peripheral input device controller channel by passing a signal through an i/o 
communications channel from said first computer system to said second computer 
system graphical user interface responsive to said receiving step. However, it would 
have been obvious to one having ordinary skill in the art at the time of the invention to 
add generating a user peripheral input device input action within said second computer 
system graphical user interface as interpreted by a second computer peripheral input 
device controller channel by passing a signal through an i/o communications channel 
from said first computer system to said second computer system graphical user 
interface responsive to said receiving step to Tuttle and Moore. One could have 
been motivated to add generating a user peripheral input device input action within said 
second computer system graphical user interface as interpreted by a second computer 
peripheral input device controller channel by passing a signal through an i/o 
communications channel from said first computer system to said second computer 
system graphical user interface responsive to said receiving step to Tuttle and Moore 
because as a user in Tuttle issues inputs that are communicated to the second 
computer and the second computer responds to the inputs with video signals, it is 
reasonable to conclude that a user's subsequent input(s) would be in response to video 
responses by the second computer as is typical when a user uses a user interface (Ul) 
of a computer. That is, when a user engages a Ul of a computer, the user's inputs are 
neither random nor disassociated from the response to the input. Therefore, it is 
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reasonable to conclude that in testing a Ul or a computer, a user's input would be 
associated in some way, either in response to or in anticipation of, a Ul response. 

Neither Tuttle nor More disclose: 

a. monitoring said second computer system graphical user interface or a pixel 
image of said second computer system graphical user interface from said first 
computer system for an expected second graphical element within a 
predetermined time interval; and 

b. signaling a failure at said first computer system if said predetermined time 
interval elapses without detecting said expected second graphical element. 

However, Tuttle discloses that at those points during playback which corresponds to the 
original command to capture visual display data, the software modules send the digital 
video signal processing unit (DVPU) the previously stored visual display signals, and 
the DVPU the captures new visual display signals from the visual display device. The 
previously stored visual display signals are compared by the DVPU to the newly 
captured visual display signals and if these signals do not match, then the DVPU sends 
an indication to the software modules on the host that an error has occurred (column 6, 
lines 31-41). Therefore, it would have been obvious to one having ordinary skill in the 
art at the time of the invention to add 

a. monitoring said second computer system graphical user interface or a pixel 
image of said second computer system graphical user interface from said first 
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computer system for an expected second graphical element within a 
predetermined time interval; and 
b. signaling a failure at said first computer system if said predetermined time 
interval elapses without detecting said expected second graphical element, 

to Tuttle and Moore. One could have been motivated to add 

a. monitoring said second computer system graphical user interface or a pixel 
image of said second computer system graphical user interface from said first 
computer system for an expected second graphical element within a 
predetermined time interval; and 

b. signaling a failure at said first computer system if said predetermined time 
interval elapses without detecting said expected second graphical element, 

to Tuttle and Moore because It is unreasonable to conclude that a user in Tuttle would 
randomly intersperse capture commands through an input sequence. In fact, the 
examiner asserts that a user would initiate a capture command where it would be most 
appropriate to capture the video results of an input or a sequence of inputs. In other 
words, a user would only issue a capture when the user knew that an input or sequence 
of inputs would result in an expected video signal, i.e. the user would not issue the 
capture command before the expected resultant video signal. However, it is also 
unreasonable to expect that the user would wait an indeterminate amount of time after 
the input or sequence of inputs to issue a capture command. Therefore, it would have 
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been obvious to include a timing mechanism in the capture command such that the 
capture command is issued before after the expected resultant video signal is 
generated but within a reasonable amount of time. 

Claims 9 and 17: Tuttle and Moore disclose the methods of claims 8 and 16 and Tuttle 
further discloses the steps of: 

a. transferring said user input action to a script stored on said first computer system 
(column 7, lines 19-30); 

b. re-executing said steps of receiving, generating, monitoring and signaling 
subsequent to said storing step under control of said stored script (column 7, 
lines 48-58).. 

Claims 10 and 18: Tuttle and Moore disclose the methods of claims 8 and 16 and Tuttle 
discloses further the steps of: 

a. providing graphical user interface language extensions commands to a scripting 
language (column 15, lines 56-68, column 16, lines 1-24); and 

b. passing said generated user input action through said graphical user interface 
language extensions from a scripting language processor to a language 
extensions processor (column 8, lines 21-26). 

Claims 1 1 and 19: Tuttle and Moore disclose the methods of claims 8 and 16 and Tuttle 
further discloses the steps of: 
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a. generating a user input action witliin said second computer system responsive to 
said second grapliical element (column 8, lines 18-26); 

b. monitoring said second computer system graphical user interface for an 
expected third graphical element within a predetermined time interval (column 6, 
lines 28-30, 34-39); and 

c. signaling a failure at said first computer system if said predetermined time 
interval elapses without detecting said expected third graphical element (column 
6, lines 39-41). 

Claims 12 and 20: Tuttle and Moore disclose the methods of claims 8 and 16 and Tuttle 
further discloses the steps of: 

a. receiving a local user input action at said first computer system within said local 
display (column 6, lines 1-4, column 36, lines 6-14); 

b. wherein said generated user input action emulates said local user input action 
(column 7, lines 12-18), 

but does not disclose depicting said second computer system graphical user interface 
upon a local display of said first computer system including said first graphical element. 
However, Moore discloses depicting said second computer system graphical user 
interface upon a local display of said first computer system including said first graphical 
element (column 3, lines 8-25). Further, Tuttle discloses that the first computer (host) 
has direct access to the digital video data of the display screen of the second computer 
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(system under test) (column 13, lines 57-59), and inputting inputs through a touch 
screen (column 36, lines 6-14). Therefore, it would have been obvious to one having 
ordinary skill in the art at the time of the invention to add depicting said second 
computer system graphical user interface upon a local display of said first computer 
system including said first graphical element to Tuttle and Moore. One could have been 
motivated to add depicting said second computer system graphical user interface upon 
a local display of said first computer system including said first graphical element to 
Tuttle and Moore because as a user in Tuttle issues inputs that are communicated to 
the second computer and the second computer responds to the inputs with video 
signals, it is reasonable to conclude that a user's subsequent input(s) would be in 
response to video responses by the second computer as is typical when a user uses a 
user interface (Ul) of a computer. That is, when a user engages a Ul of a computer, the 
user's inputs are neither random nor disassociated from the response to the input. 
Therefore, it is reasonable to conclude that in testing a Ul or a computer, a user's input 
would be associated in some way, either in response to or in anticipation of, a Ul 
response. In order for a user to issue commands in response to Ul responses on the 
second computer, it would be beneficial for the user to be able to see the display of the 
second computer on the display of the first computer. 

Claims 13 and 21 : The method of claim 8 further comprising the steps of: 

a. providing graphical user interface language extensions commands to a scripting 
language (column 15, lines 56-68, column 16, lines 1-24); and 
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b. receiving a local user input action within said local display (column 6, lines 1-4, 
column 36, lines 6-14); 

c. transferring said user input action to a script stored on said first computer system 
(column 7, lines 19-30); 

d. passing said generated user input action through said graphical user Interface 
language extensions from a scripting language processor to a language 
extensions processor for reproduction at said second computer system graphical 
user interface, wherein said generated user input action emulates said local user 
Input action (column 15, lines 56-68, column 16, lines 1-24, column 8, lines 21- 
26, column 7, lines 12-18); and 

e. re-executing said steps of receiving, generating, monitoring and signaling 
subsequent to said storing step under control of said stored script (column 7, 
lines 48-58), 

and Moore discloses depicting said computer system graphical user interface upon a 
local display of said first computer system including said first graphical element (column 
3, lines 8-25). Further, Tuttle discloses that the first computer (host) has direct access 
to the digital video data of the display screen of the second computer (system under 
test) (column 13, lines 57-59), and inputting inputs through a touch screen (column 36, 
lines 6-14). Therefore, it would have been obvious to one having ordinary skill in the art 
at the time of the invention to add depicting said computer system graphical user 
interface upon a local display of said first computer system including said first graphical 
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element to Tuttle and Moore. One could have been motivated to add depicting said 
computer system graphical user interface upon a local display of said first computer 
system including said first graphical element to Tuttle and Moore because as a user in 
Tuttle issues inputs that are communicated to the second computer and the second 
computer responds to the inputs with video signals, it is reasonable to conclude that a 
user's subsequent input(s) would be in response to video responses by the second 
computer as is typical when a user uses a user interface (Ul) of a computer. That is, 
when a user engages a Ul of a computer, the user's inputs are neither random nor 
disassociated from the response to the input. Therefore, it is reasonable to conclude 
that in testing a Ul or a computer, a user's input would be associated in some way, 
either in response to or in anticipation of, a Ul response. In order for a user to issue 
commands in response to Ul responses on the second computer, it would be beneficial 
for the user to be able to see the display of the second computer on the display of the 
first computer. 

Claim 14: Tuttle discloses a method for enabling a local system to remotely operate a 
remote computer system through a graphical user interface on said remote computer 
system by using local scripts that selectively respond to changes in graphical displays 
upon said graphical user interface of said remote computer system, comprising the 
steps of: 

a. capturing user input effected in said depiction of said remote system graphical 
user interface display (column 6, lines 22-28); 
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b. implementing through a local system command language set user input 
emulations representative of said captured user input reproduced at said remote 
computer system graphical user interface through a channel (column 15, lines 
56-68, column 16, lines 1-24); 

c. image processing said remote computer system graphical displays to detect 
changes in said graphical display upon said graphical user interface of said 
remote computer system (column 6, lines 28-30, 34-39); 

d. controlling a flow of execution of said local system through a scripting language 
having scripting commands in combination with said command language set 
(column 6, lines 31-48); and 

e. communicating between said local system and said remote computer system 
graphical user interface (column 6, lines 31-48). 

Moore discloses displaying a depiction of said remote system graphical user interface 
display on said local system (column 3, lines 8-25). Further, Tuttle discloses that the 
first computer (host) has direct access to the digital video data of the display screen of 
the second computer (system under test) (column 13, lines 57-59), and inputting inputs 
through a touch screen (column 36, lines 6-14). Therefore, it would have been obvious 
to one having ordinary skill in the art at the time of the invention to add displaying a 
depiction of said remote system graphical user interface display on said local system to 
Tuttle. One could have been motivated to add displaying a depiction of said remote 
system graphical user interface display on said local system to Tuttle because as a user 
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in Tuttle issues inputs that are communicated to the second computer and the second 
computer responds to the inputs with video signals, it is reasonable to conclude that a 
user's subsequent input(s) would be in response to video responses by the second 
computer as is typical when a user uses a user interface (Ul) of a computer. That is, 
when a user engages a Ul of a computer, the user's inputs are neither random nor 
disassociated from the response to the input. Therefore, it is reasonable to conclude 
that in testing a Ul or a computer, a user's input would be associated in some way, 
either in response to or in anticipation of, a Ul response. In order for a user to issue 
commands In response to Ul responses on the second computer, It would be beneficial 
for the user to be able to see the display of the second computer on the display of the 
first computer. 

Moore discloses a system where video data is sent from a second computer (a system 
under test) to a first computer (host or remote computer) over a communication channel 
(column 2, lines 41-59) and displayed on a display of the first computer (column 3, lines 
8-19). A user inputs, i.e. mouse and keyboard inputs, can be sent over a 
communications channel from the first (host or remote computer) to the second 
computer, system under test (column 4, lines 16-23) where the Input data Is processed 
(column 4, lines 39-50). Therefore, it would have been obvious to one having ordinary 
skill in the art at the time of the invention to add communicating between said local 
system and said remote computer system graphical user interface through a 
communication interface responsive to said flow controlling step to Tuttle and Moore. 
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One could have been motivated to add communicating between said local system and 
said remote computer system graphical user interface through a communication 
interface responsive to said flow controlling step to Tuttle and Moore in order to extend 
the range of communication between the first and second computers of Tuttle. It 
appears that Tuttle Is Intended to be used wherein the first and second computers are in 
close proximity. Adding the limitations of Moore to Tuttle could allow Tuttle to be able to 
test the software/hardware of the second computer at the first computer over a network 
over a larger distance. 

Neither Tuttle nor Moore disclose controlling a flow of execution of said local system 
through a scripting language having scripting commands in combination with said 
command language set responsive to a detection of changes during said image 
processing step. However, it would have been obvious to one having ordinary skill in 
the art at the time of the invention to add controlling a flow of execution of said local 
system through a scripting language having scripting commands in combination with 
said command language set responsive to a detection of changes during said 
image processing step to Tuttle and Moore. One could have been motivated to add 
controlling a flow of execution of said local system through a scripting language having 
scripting commands in combination with said command language set responsive to a 
detection of changes during said image processing step to Tuttle and Moore 
because as a user in Tuttle issues inputs that are communicated to the second 
computer and the second computer responds to the inputs with video signals, it is 
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reasonable to conclude that a user's subsequent input(s) would be in response to video 
responses by the second computer as is typical when a user uses a user interface (Ul) 
of a computer. That is, when a user engages a Ul of a computer, the user's inputs are 
neither random nor disassociated from the response to the input. Therefore, it is 
reasonable to conclude that In testing a Ul or a computer, a user's input would be 
associated In some way, either in response to or in anticipation of, a Ul response. 

Claim 15: Tuttle and Moore disclose the method for enabling a local system to remotely 
operate a remote computer system through a graphical user interface on said remote 
computer system of claim 14 and Tuttle further discloses the steps of: 

a. storing said scripting commands into a storing means (column 7, lines 23-26); 

b. inserting a command from said command language set into said storing means 
(column 15, lines 56-63, column 29, lines 23-28); and 

c. executing said inserted stored command (column 29, lines 29-39). 

Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to JOHN M. HEFFINGTON whose telephone number is 
(571 )270-1696. The examiner can normally be reached on Mon - Fri 8:00 - 5:30 EST. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Weilun Lo can be reached on (571) 272-4847. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

JMH 
11/6/08 



/Ba Huynh/ 

Primary Examiner, Art Unit 2179 



